On the Complexity of Policy Iteration
نویسندگان
چکیده
Decision-making problems in uncertain or stochastic domains are often formulated as Markov decision processes (MD Ps). Pol icy iteration (PI) is a popular algorithm for searching over policy-space, the size of which is exponential in the number of states. We are interested in bounds on the complexity of PI that do not depend on the value of the discount factor. In this paper we prove the first such non-trivial, worst-case, upper bounds on the number of iterations required by PI to converge to the optimal policy. Our analysis also sheds new light on the manner in which PI progresses through the space of policies.
منابع مشابه
An improved infeasible interior-point method for symmetric cone linear complementarity problem
We present an improved version of a full Nesterov-Todd step infeasible interior-point method for linear complementarityproblem over symmetric cone (Bull. Iranian Math. Soc., 40(3), 541-564, (2014)). In the earlier version, each iteration consisted of one so-called feasibility step and a few -at most three - centering steps. Here, each iteration consists of only a feasibility step. Thus, the new...
متن کاملSearch Based Weighted Multi-Bit Flipping Algorithm for High-Performance Low-Complexity Decoding of LDPC Codes
In this paper, two new hybrid algorithms are proposed for decoding Low Density Parity Check (LDPC) codes. Original version of the proposed algorithms named Search Based Weighted Multi Bit Flipping (SWMBF). The main idea of these algorithms is flipping variable multi bits in each iteration, change in which leads to the syndrome vector with least hamming weight. To achieve this, the proposed algo...
متن کاملSearch Based Weighted Multi-Bit Flipping Algorithm for High-Performance Low-Complexity Decoding of LDPC Codes
In this paper, two new hybrid algorithms are proposed for decoding Low Density Parity Check (LDPC) codes. Original version of the proposed algorithms named Search Based Weighted Multi Bit Flipping (SWMBF). The main idea of these algorithms is flipping variable multi bits in each iteration, change in which leads to the syndrome vector with least hamming weight. To achieve this, the proposed algo...
متن کاملThe Effect of on tax professionals ' perception of tax complexity on tax compliance behavior
Today, the role of tax professionals has become an important issue in tax policy due to more complex and ambiguous tax laws. For this reason, the study of the activities of tax professionals is important for two reasons. Firstly, Taxpayers use the services of tax professionals to meet their tax obligations. Secondly, tax professionals, more than taxpayers, experience the issue of tax complexity...
متن کاملCorrector-predictor arc-search interior-point algorithm for $P_*(kappa)$-LCP acting in a wide neighborhood of the central path
In this paper, we propose an arc-search corrector-predictor interior-point method for solving $P_*(kappa)$-linear complementarity problems. The proposed algorithm searches the optimizers along an ellipse that is an approximation of the central path. The algorithm generates a sequence of iterates in the wide neighborhood of central path introduced by Ai and Zhang. The algorithm does not de...
متن کاملMultiple solutions of a nonlinear reactive transport model using least square pseudo-spectral collocation method
The recognition and the calculation of all branches of solutions of the nonlinear boundary value problems is difficult obviously. The complexity of this issue goes back to the being nonlinearity of the problem. Regarding this matter, this paper considers steady state reactive transport model which does not have exact closed-form solution and discovers existence of dual or triple solutions in so...
متن کامل